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1. Introduction 

Let X be a genus g > 2 proper and smooth curve over an algebraically closed field k of 
characteristic p 2. We let Jx denote the Jacobian of X. We choose once for all a theta 
characteristic kq over X and we let G denote the corresponding symmetric principal divisor on 
Jx representing the canonical polarization of Jx- The coarse moduli space Sx of (S-equivalence 
classes of) rank 2 and trivial determinant semi-stable bundles over X identifies with a normal 
subvariety of dimension 4:{g — 1) — 1 of the linear space |20| = p2S-i q£ effective divisors linearly 
equivalent to 20 on Jx- Furthermore, its semi-stable boundary identifies with the Kummer 
variety Kx, which is canonically embedded in |26|. 

Assume that k has positive characteristic p > 3 and denote by Xi (resp. Jxi, Sxi,---) the 
Probenius twist of X (resp. Jx, Sx,---)- The relative Probenius F : X ^ Xi is a fc-morphism 
that induces by pull-back a morphism Vj : Jxi — »■ Jx classically called Verschiebung, etale when 
X is ordinary and closely related to the multiplication by p. It analogously induces a rational 
map Vs : Sxi — Sx, called the (generalized) Verschiebung, that is generically etale for an 
ordinary curve and restricts to the morphism Kxi Kx induced by Vj on the semi-stable 
boundary. There are stable vector bundles E over Xi such that F*E is not semi-stable and the 
corresponding set of points in Sxt is precisely the base locus BiVs) of Vs- 

In genus 2, Sx — \2Q\ and Kx is a quartic surface, the Kummer surface, with 16 nodes 
corresponding to the 16 points of Jx[2] and forming a so-called 16q configuration in |2G| = P^. 
The Verschiebung Vs is given by degree p polynomials. The base locus of Verschiebung is finite 
and we know from the work of B. Osserman [Ossj (see also [LaP| ) that it is reduced for a general 
curve, of length 2{p^ — p)/S, and that the indeterminacy of Vs can be resolved by a single 
blowing-up of Sxi at B{Vs)- As a consequence, has degree (p^ -|- 2p)/3- 

In genus 2 and characteristic 3, Y. Laszlo and C. Pauly have given a nice description of 
Verschiebung |LP2l Section 6]. Consider the family of rank 2 and trivial determinant semi-stable 
vector bundles parameterized by the blowing-up Bl2(Jx) of Jx at its 2-torsion (viewed as a 
parameter space for pairs {L, [(/?]), L in Jx, [v] in ^H^{X, ®ujx)) and defined by the exact 
sequences 

^ £l, m ^ F^L ^ L®ujx^^^ 

where L also denotes the p-twist of L (in such a way that F*L = L^). The moduli property thus 
induces a morphism Bl2(Jx) Sxi that is proved, using fine properties of Kummer surfaces. 
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to factor through an embedding a : Kx — > Sxi such that Vs coincides with the polar map of 
a{Kx)- In particular, there is an equality of divisors 

VsHKx) = Kx,+2a{Kx) 

in Sxi and B{Vs) is precisely the singular locus of a{Kx)- 

In |Duj . using Prym varieties, we compute the equations defining Vs in characteristic p = 3, 5 
or 7, recovering the results of |LP2] . We can thus check that for a general genus 2 curve, there 
is a normal and integral surface H of degree 2{p — 1) in Sxi (the Zariski closure of the set 
{V^^{Kx)y of stable bundles E such that F*E is strictly semi-stable) such that the equality of 
divisors 

VsHKx) = Kx,+2H 

in Sxi holds in Sxi- Also, we could thus test on many curves (with no exception encountered) 
the following facts : B{Vs) is contained in the 0-dimensional singular locus Sing(^) of H, there 
are singular point of H where Vs is defined, these points are mapped onto the singular locus 
Sing(i^'x) of Kx and the inverse image of Sing(Kx) is 1-dimensional. 

In this work, we look at the Zariski closure H of the set {Vg^{Kx)y of stable rank 2 and 
trivial determinant bundles E over Xi such that F*E is non-stable for a general curve of genus 
5 > 2 in characteristic p > 2{g — 1). It is dominated by a proper scheme Ti. whose study boils 
down to the study of the Hilbert scheme H of rank 2 and degree subbundles of For a 

point {E, a) of H, the quotient fitting in an exact sequence 

^ ^ ^ F,Ox -^Qe^O 

is stable and the forgetful map {E, a) i-^ is a closed immersion if p > 2g. In this case, Ti 
has dimension at most 2{g — 1) (its expected dimension as a Hilbert scheme) and smoothness is 
therefore given by the vanishing of an . The Prym varieties associated to double etale covers 
of X provide us with (a finite number of) smooth points. 

In genus 2, a result of Tong Jilong ensures that the divisor 0^ associated to the sheaf of 
locally exact differential forms is smooth. Pulling-back the canonical exact sequence 

(1.1) ^ Oxi ^ F,Ox ^B^O 

by an injection L ^ B where L is a degree line bundle over Xi, provides with essentially every 
strictly semi-stable points of 7i. 

We derive from these facts our main result : 

Theorem 1.1. Let X he a general genus 2 curve over an algebraically closed field with charac- 
teristic p > 3. There is an integral and normal surface H in Sxi, with degree 2{p— 1), such that 
the equality 

Vs\Kx) = Kx,+2H 

holds scheme-theoretically. 

All schemes will be (locally) noetherian over a base field k. Unless otherwise specified, a point 
of fc-scheme S* is a closed point (i.e., a /c-point). We will let Sch/Zc denote the category of schemes 
over k and Sets denote the category of sets. 



FROBENIUS INVERSE IMAGE OF THE SEMI-STABLE BOUNDARY 



3 



If Y and Z are schemes over k, we let qy (resp. qz) denote the first (resp. the second) 
projection Y X Z ^ Y (resp. Y x Z ^ Z). 

When M (resp. M') is a coherent sheaf over a scheme T, £nd{M) (resp. £ndQ{M), resp. 
Ti.om{M, M')) is the sheaf of germs of ©T-hnear endomorphisms of M (resp. traceless endomor- 
phisms of M, resp. homomorphisms from M to M'). Global sections form the space End(M) 
(resp. Endo(M), resp Hom(M, M')). The dual of a vector bundle M over T is denoted by M"^. 

We will say that two elements /, /' of Hom(M, M') coincide if they are equal up to multipli- 
cation by a non-zero scalar. 



2. Preliminaries 



Let X be a proper and smooth curve of genus g > 2 over an algebraically closed field k of 
characteristic p 2. 

2.1. Moduli spaces. In the sequel, we will always denote by Sx (resp. Nx) the coarse moduli 
space of rank 2 and trivial determinant (resp. degree 0) semi-stable vector bundles over X. 
These have dimension 4:{g — 1) — 1 and 5{g — 1) respectively. 

We denote by Sj^ (resp. N^) the stable locus of Sx (resp. Nx) and by S'^ (resp. Njf) 
the semi-stable boundary of Sx (resp. Nx). The semi-stable boundary of Sx (resp. of Nx) 
identifies with the Kummer variety Kx of X and it is precisely the singular locus of Sx (resp. 
of Nx) unless g = 2. Whenever a scheme T parameterizes a family £ of rank 2 and trivial 
determinant (resp. degree 0) over X, we let denote the inverse image of Sx (resp. A^^) by 
the map T Sx (resp. T — > Nx) induced by the moduli property. Similarly, T** will denote 
the inverse image of S^ (resp. N^). 

It is well-known that Sx and Nx are normal (even smooth when g = 2) and projective varieties 
over k and that there is a finite etale morphism t : Sx x Jx Nx (mapping (E, L) io E ® L) 
of degree 2^^. Taking the determinant of a vector bundle gives a morphism det : Nx — > Jx and 
letting [2] : Jx ^ Jx denote multiplication by 2, the diagram 

Sx X Jx ^ Jx 

det 

(2.1) Nx Jx 

is cartesian. Recall that, at a stable point E of Nx (resp. Sx), the tangent space TeNx (resp. 
TeSx) at E is canonically isomorphic to H^{£nd{E)) (resp. H^{£ndo{E))) and the tangent 
map of det : Nx —>■ Jx is the trace map Tr : H^{£nd{E)) H^{Ox)- 

When k has characteristic p, we let denote the moduli space 

Mx{p-2, {p-l){g-l)) 

of rank p — 2 and degree {p — l){g — 1) semi-stable vector bundles over X. It is proper and 
normal of dimension (p- 2)2(^-1)-! and its tangent space at a stable point Q is canonically 
isomorphic to H^{£nd{Q)). 
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2.2. An upper bound for slopes of rank r subbundles of F^rj. We apply a construction 
of |JRXYj relying on adjunction and a theorem of P. Cartier (see [Kal Theorem 5.1]) to prove 
the following key-lemma 

Lemma 2.1. Let X be a genus g > 2 curve over k and let rj be a degree d line bundle over X . 
For all 1 < r < p, a rank r subbundle M of F^r] has slope 

{r-l){g-l)+d 

^ < t^riV) = 

p 

Furthermore, there is equality if and only if F*M has a filtration 

{0} = MrC Mr-i C ... C Ml C Mo = F*M 
with Mi/Mi+i = ujf <S) V for all < i < r - 1. 

Proof. The case r = 1 comes directly from adjunction formula. If M is a rank r > 2 subbundle 
of F^,L, we write Mq = F*M and we let Mi denote the kernel of the non zero map ipo : Mq = 
F*M —I- Tj corresponding to M F^r/ by adjunction. There is an exact sequence 

^ Ml ^ Mo ^ Mo/Ml ^ 

over X and deg(Mo/Mi) < d with equality if and only if Mq/Mi = rj. The second fundamental 
form of the Cartier's connection V : Mo — > Mo ® ux associated to this exact sequence is a Ox- 
linear map (fi : Mi — > (Mq/Mi) ®ujx that is not zero for otherwise Mi descends to a subbundle 
of Mo, hence of F^^rj, and the composite Mi Mq Ox does not vanish by functoriality of 
adjunction. 

If r = 2, Ml is a line bundle and one has degMi < deg(Mo/Mi) + 2{g - 1) <2{g -I) +d 
with equality if and only if Mi = Mq/Mi ® ^x, hence 

1 . l deg(Mi) + deg(Mo/Mi) ^ 

/i = -/x(Mo) = < iX2{m 

p p p 

with equality if and only if Mq/Mi — f] and Mi = ujx ® V- 

If r > 3, we let M2 denote the kernel of fi, a rank r — 2 subbundle of Mo and we let 
(/?2 : M2 M0/M2 (8) uJx denote the second fundamental form of V associated to the exact 
sequence 

^ M2 ^ Mo ^ M0/M2 
For the same reason as above, ip2 does not vanish and we construct inductively a filtration 

{0} = MrC Mr-i C ... C Ml C Mo = F*M 

of Mq and a family of Ox -linear maps 

<Pk-Mk^ (Mo/Mfc) ivx 

where 

Mk = ker(v9fc-i : Mfe_i ^ (Mo/Mfc_i) (g) cox) 
and ifk is the second fundamental of V associated to the exact sequence 

^ Mfe ^ Mq ^ Mo/Mfc ^ 
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By construction, the map ^pk induces an injection {Mk/Mk+i) ^ (Mk-i/Mk) (^wx which yields 
by induction an inequahty 

T — 1 V — 1 

^ = -/x(Mo) = — V deg(Mfe/Mfc+i) < — V(2(5 - l)k + d) = ^,(77) 

k=0 fc=0 

with equahty if and only if M^/Mk+i = oo\ 77 for all < A; < r — 1. □ 

Remark 2.2. It immediately follows that a rank r quotient sheaf M' of F*?7 has slope 
^ I {2p-r-l){g-l)+d , , (p-r)(g-l) 

p p 

In the sequel, we simply write fir (resp. I'r) for fir{Ox) (res. fr(Ox))- 

Remark 2.3. It follows from Lemma [2?T] that F^rj is stable. When one tries to provide an upper 
bound fir(W) for rank r subundles of F^W where is a stable rank n bundle over X, one can 
proceed analogously to define a filtration 

{0} = MrC Mr-i C ... C Ml C Mo = F*M 

in the same way as above. In particular, there is a series of inclusions 

M^„i C (M^„2/Mr-i) ® C ... C (M1/M2) ® w^^"^ C (Mo/Ml) u;^"^ CW ® uj^'^^ 

However, since we are dealing with higher rank vector bundles, one is confronted to the fact that 
a non zero map needs not be of maximal rank. Still, if rk(Mfc/Mfc+i) < rk(Mfc_i/M/c), there is 
a quotient bundle W — > W" with kernel W' such that the composite 

Mk/Mk+i C {Mo /Ml) (g) uj^ C W (g) uj'^ ^ W" (g) uj'^ 

is zero. Letting M' denote the kernel of the composite M C F^:W this is a subbundle 

of F^W' and we can analogously define a sequence 

{0} = M;, C M;,_i C ... C M( C M^ = F*M' 

The point is then to notice that = Mi for all i > A: and proceed by induction on n to prove 
that r < p. The stability of W thus induces the stability of (see |Sun| for an another proof 
much in the same spirit and a generalization of the argument for higher dimensional varieties). 

2.3. Recollections on the divisor @b- Assume that k has characteristic p > 3 and recall that 
the differential d : Ox — > ^^x kills all p-powers and induces the two short exact sequences (II. ip 
and 

(2.2) 0^5^ F^iux ^ wxi ^ 

of vector bundles over Xi, where c is the Cartier's operator (see |Ral Section 4]). The bundle B 
of locally exact differential forms has rank p — 1 and slope g — 1. Taking tj = Ox in the lemma 
12.11 we find that B is stable (seejJo] for another argument) and it provides a filtration 

(2.3) {0} = BpC Bp^i C .... CBi = F*B C Bq = F*F^Ox 

with Bi/BiJ^i = for all < i < p — 1. (recall that the filtration (12.31) of Bi was obtained 
in [Ral Section 4] using the interpretation of Bi as the augmentation ideal of the {p — l)-th 
thickening of the diagonal in X X). 
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Recall that the bundle B is endowed with a non-degenerate skew-symmetric bilinear form 

(2.4) T : B ® B Loxi (or, equivalently, T : A^B wxi) 
Because F*F^,Ox = Bq = Ox © Bi, there is an isomorphism 

F^Ox «) F^Ox = F,Ox © F^Ox © B 

where the projection on the first factor is the canonical multiplication map. Thus, the subbundle 
A^F*Ox C F^Ox (^F^,Ox actually lies in F^Ox B. Also, it is tautological that the composite 
A^F^Ox F^Ox 0B ^ B(S)B factors through A^B ^ 5 (g) 5 and since there is an exact 
sequence 

0^ B ^ A^F^Ox A^-B ^ 
we find that the extension class of A^F^Ox in Ext^(A^i?, B) is the image of the extension class 
of F^Ox in Ext^(S, Ox) via the map 

Ext^i?, Ox) ^ Ext\B ® B, B) ^'^^^^^^ Ext\A^B, B) 

Now, consider the exact sequence (|2.2I) and pull-it back by T : A^B —* oJXi- Because the 
extension class of F^oox in Ext^(u;xi, ^) is that of F^^Ox in Ext^(i?, Ox) via the isomorphism 
B — > B^ (g) u)Xi deduced from T, there is a map 

(2.5) f : A^F^Ox ^ F^ujx 
fitting in the commutative diagram with exact rows and columns 
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0^ B ^ A^F^Ox 


^ A^B 
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where T is the kernel of T. 

In [Raj, M. Raynaud proves that the set 

{L G JxjHomxi(L, 5) 7^0} 

is the support of a well-defined divisor 0^. The latter is algebraically equivalent to {p — 1)0 
and totally symmetric (in the sense of Mumford), hence comes from a divisor on the Kummer 
variety Kx^ of Xi. In case X is an ordinary curve, Qb does not go through the origin of Jx^ 
(in particular H^{B) = H^{B) = 0). We recall a few results on Gb that Jilong Tong, a former 
student of M. Raynaud, obtains by degeneration technics (PhD Thesis |Toj). Let us first set the 
following 

Definition 2.4. Let TIb be the Quot scheme representing the functor Tlilbi^o{B) from the 
category of schemes over k to Sets and defined by 

{rank p — 2 quotient sheaves q^-^ B ^ Q ^ 
over Xi X T, flat over T, W ~ 

with degQt=p-iyte T{k) J 
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The universal line subbundle over Xi x Hb induces a morphism TCb ^ &b Jxi that can 
be seen as a "projective" bundle over 6^ (meaning that the fiber above a point L of 6b is 
isomorphic to PHomxj(L, i?)^). 

Theorem 2.5 (|Toj). Let X be a general proper and smooth curve of genus g > 2 over an 
algebraically closed field k. 

(1) If g = 2, the divisor @b is smooth and the map TCb Qb is an isomorphism (Theorem 
3.2.2, p. 60). 

(2) If g >'i, the divisor Qb is normal (Theorem 3.2.3, p. 60). 

(3) If g > 2, JxiIp] \ {0} is the whole set of points of finite order in Qb and Qb is smooth at 
each of them (Proposition 3.3.2.1, p. 63). In particular, if L is an order p line bundle over Xi, 
dimHom(L, B)) = 1 (Corollary 2.3.2, p. 50). 

3. Rank 2 and degree subbundles of F^Ox for g>2 

Prom now on, we assume that the base field k has characteristic p > 2{g — \) and that the 
curve X is general. In particular, it is ordinary and we have a finite etale group scheme 

(3.1) G := Jx, [p]red = ker(Vj : Jx, ^ Jx) = {I^lpTL)' 

(it is well known that this statement holds for any ordinary abelian variety of dimension 5 > 1). 
3.1. Some definitions. Our main object of study will be the following 

Definition 3.1. Let r/ be a degree d line bundle over X with d < and let Hr] be the Quot-scheme 
representing the functor Ti.ilb2,o{F^r]) from Sch/k to Sets and defined by 

{rank p — 2 quotient sheaves g^CjC-^*^) Q 1 
over Xi X T, flat over T, W ~ 

with deg Qt = {g-l){p-l)+dyte T{k) J 

where qx-i : Xi x T ^ Xi denotes the flrst projection. 
\i Tj = Ox , we let TL denote the Quot-scheme Hox ■ 

For a closed point {E, a) of 7Y, we let Qe denote the quotient sheaf fltting in the exact 
sequence 

(3.2) ^ ^ F^Ox -^Qe^O. 

By adjunction, there is a non-zero map ad(a) : F*E Ox that factors through an injection 
SA : 0(— A) ^ Ox, where A is an efl'ective (and possibly trivial) divisor. If A is has maximal 
degree among those with this property, the map F*E — > 0{—A) is surjective and there is an 
exact sequence 

(3.3) ^ 0(A) O F*(det E) F*E 0{-A) 

One checks that the linear equivalence class 0{A) of A is uniquely deflned and in case 0{A) is 
non-trivial, we call its degree the degree of Frobenius-destabilization of E. 

In application to the Hilbert scheme H, the lemma[2]T]has the following immediate consequence 
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Corollary 3.2. Assume that {E, a) is a point of 7i and that F*E fits in an exact sequence as 
above. The second fundamental form 

: 0(A) O F*(det E) 0(-A) ujx 

of C artier 's connection associated to this exact sequence does not vanish. 
In particular, H^{X, 0{K - 2A) det{F*E)) / and < deg(A) <g-l. 

If A = 0, F*E is semi-stable and one has Homxi(^, F^Ox) = Yiomx{F*E, Ox) by adjunc- 
tion. We define 

712 ■■= {{E, a) in H] F*E ^ Ox ® Ox}- 

Remark 3.3. Assume that F*E is semi-stable and let e in H^{X, F*{detE)) denote the ex- 
tension class of (13. 3p . Letting ifi denote as above the second fundamental form of Cartier's 
connection associated to this exact sequence, a straightforward cocycle computation yields that 
the cup-product e U ipi in H^{uJx) vanishes. 

If A is non trivial, it follows from |LaSl Satz 2.5] that the extension (13. 3|] is non-split and one 
has Romx.iE, F^Ox) = H^{X, 0(A)). The universal inclusion s : 0{-A) Ox over X x |A| 

F s 

gives rise to an inclusion F*0(-A) F^Ox over Xi X |A| and the universal property of H 
yields a morphism 

'^C'(-A) X |A| ^ 

One checks that this is a closed immersion and we denote Wi^i its image in Ti. We check similarly 
that if A and A' are two effective divisors with A — A' effective, there is a series of inclusions 



Recall the generalized Nagata-Segre theorem (relying on a result of Hirchowitz |Hir| . see also 
(LaPl Theorem 2.1]) : 

Theorem 3.4. For any rank r and degree 6 vector bundle W over any smooth curve X of genus 
g and for any integer n with 1 < n < r — 1, there is a subbunble E C W such that the inequality 

r — n c 

^^{E)>^l{w) (g-i)- — 

r rn 

holds, where e is the only integer with < e < r — 1 and e + n{r — n){g — 1) = n5 mod r. 

Corollary 3.5. For any line bundle r] with degree 1 — g < d < 0, the Quot-scheme Tirj is non 
empty. 

Proof. Taking W = F^rj and n = 2, we obtain e = 2[g — 1 + d) and considering our hypothesis 
on p, g and d, e = 2{g — 1 + d). Now, the theorem provides us with a rank 2 subbundle of F*?7 
with slope 0. □ 

On the one hand, the universal subbundle S^^ in the universal exact sequence 

^ ^ Sr, ^ q*x,{F.^) ^ Qn ^ ^ 
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over Xi X 7^^ obviously induces a forgetful morphism 

(3.4) hs^ : Hr, ^ Nx, 

via the moduli property. Using the morphism t : Sxi x Jxi ^Xi, we can consider the fiber 
product Tir) XNx-^ i'^Xi x Jxi)- The map Kjj x^Vxj (-S'xi x Jxi) — ^ Ti-r) is etale of degree 2^^ and 
there is a map Ti^ x^Vxj ('S'xi x Jxi) Sxi x Jxi above hg^. 

On the other hand, the group G naturally acts upon Hr) for any r). Indeed, for any 7 in G, use 
the canonical isomorphism F^rj (8) 7 — ^ F^rj and define r-y as the automorphism of Tin defined by 

(3.5) r^{ST) = St ^ q*x,{l) 

for any such 7 and for any T-point of 7i^. This action is obviously free and it defines an etale 
quotient map Hrj 'Hrj/G of degree p^. This action can be lifted to a free action of G upon 
T^T] ^Nx^ i'^Xi X Jxi) and the quotient map is similarly etale finite of degree p^. 

Definition 3.6. We define 



Hr, 



Hr, xjvx, {Sx^ X Jx,) /G (resp. H := H xn^, {Sx, x Jx,) /G). 



We let 7^2 denote the set of points in Ti such that F*E is split isotypic. 

We let 'H\A\ denote the closed subscheme of 7i corresponding to for any effective divisors 
A as above. It is the image of the induced closed immersion Ho^-A) x |A| ^ H. 

A point in 7^|a| thus can be seen as a triple {E, ^, a) where £^ is a semi-stable rank 2 vector 
bundle with trivial determinant, where ^ is a degree line bundle over X and where a, defined 
up to a multiplicative scalar, is an embedding E ^ F^{0{—A) (g) ^). 

Definition 3.7. There is an induced map Hr, — > Sxi x Jx and we define 

hr, : Hr, — > Sxi and : Hr, — ^ Jx (resp. h :H ^ Sxi and V :H ^ Jx) 

as its composites with the projections. 
We let 

H := im{h : H ^ Sx,) 

denote the scheme-theoretic image of H {H^ and H^^ will denote respectively its stable and its 
strictly semi-stable loci) and 

H2 := {E in H'\F*E ^ r r, r in Jx[2]} 

Lemma 3.8. Assume that E is a stable vector bundle over Xi with rank 2 and trivial determinant 
(viewed as a point of Sxi)- Then, 

(1) the bundle E lies in H if and only if F*E is not stable . 

(2) the bundle E lies in the image ofH\A\ for some non trivial A if and only if F*E is unstable. 

(3) If F*E is semi-stable, then h~^{E) contains at most 2 (opposite) points unless E lies in 82- 
In this case, h~^{E) ^ with V(h-^{E)) C Jx[2]. 

In any case, the image ofV{E, ^, a) through the composite Jx — > Kx C Sx is Vs{E). 

(4) If F*E is unstable, the fibre h^^{E) identifies with the set Sym°'(X) of degree d effective 
divisors over X , where d is degree of Frobenius-destabilization ofE. 
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The map V restricts on the fibre h ^{E) to the usual map Sym'^(X) — > Jx (defined up to 
translation). 

Proof. (1), (2) and (3) immediately follow from the adjunction formula. 

As for (4), assume that E is & stable point of H lying in the image of 7^|a|- A point {E, ^, a) 
lies in the fibre h~^{E) if and only if F*E fits in an exact sequence 

^ 0(A) (g) ^ F*E ^ ^ (g) 0{-A) 

for some non-trivial effective divisor A. Then, for any effective divisor A' of degree d, there is a 
surjective map 

F*E 0(A - A') ^ ^ O(-A') 

giving rise to an injective map E ^ F*{^ ^ 0{—A')) *^ ^ ^ \ Calling this map a', 

{E, ^ (g) 0{A — A'), a') is a point of h^^{E). Notice that if A and A' are two distinct linearly 
equivalent divisors, the map a and a' do not coincide. 

Conversely, if {E, a') is an other point in h~^{E) fitting in an exact sequence 

^ O(A') ® i'-^® F*E ^ ® O(-A') ^ 

the composite 0{A) (g) ^^^0 F*E ^ 0{-A') is necessarily and 0{A) ® ^"^ is a line 
subbundle of 0{A') ® By symmetry, one obtains 0{A) (g) = O(A') ^'"^ Since the 
map V is just the forgetful map {E, ^, a) i— > ^, this concludes the proof. □ 

3.2. The "second fundamental form" map. In the sequel, we use the notations of Subsection 
[231 Recall that we let Bq (resp. Bi) denote F*F^Ox (resp. F*B) and ^0:^0^ Ox (resp. 
tpi : Bi ^ u!x) denote the evaluation map (resp. the second fundamental form of Cartier's 
connection V : Bq ^ Bq <g ujx associated to the exact sequence Q ^ Bi Bq Ox — ^ 0). 
Notice that the adjoint of V'l is the injection B ^ F^uJx of the exact sequence (|2.2I) . Eventually, 
we will use the map T : A'^F^,Ox F^ujx- Notice that pulling back by Frobenius splits the 
middle row in the diagram (|2.6p and that the map ad(T) : F*{K^F*Ox) —>■ wjf is nothing but 
the projection on Bi followed by tpi : Bi ^ uJx- 

Proposition 3.9. Let {E, a) be a point of 7i and assume F*E fits in an exact sequence (13.31) . 
The adjoint ad(<I>i(£')) : F*{detE) — > wx of the composite 

^i{E) : A^E ^ A^F.Ox ^ F,ux 
coincides (up to a multiplicative scalar) with the composite 

F*{detE) ^ F*{detE) (g 0{A) ^ 0{-A) ^ uJx 

Proof. Take a point {E, a) in Ti and pull back the exact sequence (13. 2|) back to X by Frobenius. 
Using the notations of the proof of Lemma [2m let Eq (resp. (po) denote the Frobenius inverse 
image of E (resp. the adjoint of a). By the very definition of adjunction, 930 is the composite 

Eq^F*F,Ox^Ox. 
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Since F*a induces an injective map pi : 0(A) ® F*{(letE) — > Bi and since F*a commutes 
with the canonical connections on Eq and Bq, the composites 

(3.6) 0(A) ® det Eq ^ Bi ^ ux and 0(A) ® det Eq ^ 0(-A) ®ujx^(^x 

coincide. Taking Frobenius inverse image and second exterior powers commute and F*{Is?a) 
agrees with the second exterior power of 

F*a = (v?o, F*aB) : Eq ^ Ox ® Bi 

where as denotes the composite map E F^Ox B. A local computation yields the decom- 
position 

F*{K^a) = (pi, F*{K^aB)) : det£;o Bi® A^Bi 

where pi is the composite detE'o C'(A) (8) det£^o -Bi. Noticing that the adjoint of ^i{E) 
is the composite 

det So = F*{A'E) F*{A'F*Ox) ^ ux 

concludes the proof. □ 



If one writes a line bundle L over X under the form ujx <X) 0{—xi — ... — X2g-2), where the Xj's 
are points of X, one has 



Hom(L,a;x) = H\0{xi + ... + X2g-2)) 



Consider the morphism 



defined by xi + ... + X2g-2 ^ '^x ® 0{—xi — ... — X2g-2) and call B{Jxi) and B2{Jx) respectively 
the cartesian products 

B{Jx,) Sym2(9-i)(X) B2{Jx) — 



Vj 



and 



Syui'^(9-l\X) 



Jx 



Jx 



[2] 



Jx 



It is clear that B{Jxi) represents the functor from Sch/fc to Sets defined by 

{rank p — 1 quotient sheaves g^^(F*a;x) 
over Xi X T, flat over T, 
with deg7^t = (j)+l){g- 1) Vt G T{k) 

and since any T-point {£t, «t) of T-L induces a morphism 

det£:T q*x^{A^F,Ox) q*x,{F.0Jx) 

that does not vanish at any closed points of T, there are natural morphisms 

#i:H^i3(Jxi) and $i : H ^ S2(Jx) 
We will refer to these maps as the "second fundamental form" maps. 
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It is classical that Sym^^^~^\X) is smooth over fc, of dimension 2{g — 1) and so are B{Jxi) 
and B2{Jx)- In particular, in genus 2, the map Sym^(X) Jx (resp. S2(Jx)) agrees with the 
blowing-up of Jx at the origin (resp. the blowing-up Bl2(Jx) of Jx along its 2-torsion). 

Proposition 3.10. In genus 2 and characteristic 3, the "second fundamental form" map 
is an isomorphism. 

Proof. It is equivalent to prove that $1 : 7^ ^ B{Jx-i) is an isomorphism. Take a point {E, a) 
of Ti. The quotient bundle Q^; is a line bundle (Lemma 13. 121 below) that is isomorphic to 

det(F,Ox) ® (det^)-i ^ ® {detE)'^ 

and there is an exact sequence 

Q^E ^ F^Ox cjxi ® (det^)-^ ^ 

Because T : I\?B — > ujx^ is an isomorphism in characteristic 3, the map T : A^(F^,C'x) F^ujx 
also is an isomorphism and one can identify the maps ^i{E) : h?{E) F^uJx and A^(a). 

But taking second exterior powers in the short exact sequence above yields an exact sequence 

^ K^{E) A^[F^Ox) -^E^LOx,^ det E~^ 

Because S«)(det^)-i ^ E'^ , we find that the two exact sequences correspond to each other by 
Serre's duality. □ 

Remark 3.11. This is actually the construction of [LP21 Section 6]. In particular, we find in 
that case that TC is smooth and that $1 is an etale map. 

3.3. A closed immersion via the quotient bundle. In rank 2, the lemma 12.11 has the fol- 
lowing other consequence. 

Lemma 3.12. Let {E, a) be a point ofH^. Then the quotient bundle fitting in the exact sequence 

0^ E ^ F^rj^QE,^^0 

is a stable vector bundle with rank p — 2 and degree {p — 1)(5' — 1) -|- d, where d = degr/. 
In particular, the moduli property of defines a forgetful morphism 

(3.7) hQ-.n^ 

Proof. Assume first that Qe^t] has torsion. Then, the kernel of the map F^^rj — > Q"^^ to the 
torsion free part of Qe^t/ has rank 2 and slope at least 1/2. But 

/X2(r?) = {g + d- l)/p <{g- l)/p < 1/2 

and this is a contradiction. If p = 3, Q^;^^ is a line bundle and it is stable. If p > 5, let M be a 
quotient bundle of QE,r} with rank r < p — 3. It is also a quotient of F^rj and its slope fj, satisfies 
. {p + 2){g-l) + d ^ {p-l){g-l) + d 

P P ^ 

for all p > 5 and Q^;^ is again stable. □ 
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The forgetful map Hq maps a T-point {£t, ar) of 7i to the T-point Qt of Al^^, where Qt 
fits in the exact sequence of vector bundles 



over Xi X T. 



Proposition 3.13. Assume that p > 2g. Then, the morphism hg : TC —>■ A4^^ is a dosed 
immersion. 

Proof. One only needs to show that, for any point {E, a) in H, Hom(F*Ox, Qe) has dimension 
1. Indeed, we can apply the functor Hom(— , Qe) to the exact sequence (13. 2p and the connecting 
morphism 

Rom{E, Qe) ^ Ext\QE, Qe) 

(that agrees with the tangent map Th^ : TeTC Tq^M.^^ at E) is injective in this case. Using 
adjunction and relative duality, there is an isomorphism 

Homxi(F*Ox, Qe) = HomxK"', F*Qe) 

Lemma 3.14. Let {E, a) be a point ofTC. With the notations of Lemma \3.9[ define the torsion 
sheaf 

k<i>i{E) - coker(ad($i(£')) : detE'o ujx) 
over X . The exact sequence (|3.2p over Xi gives rise to an exact sequence 

0^B2^ F*Qe ^ k^,(E) ^ 
over X (where B2 = ker(^i : Bi — > lox), see Subsection W. 



Proof. Consider the exact sequence (|3.3I) for some effective divisor A associated to [E, a). Let- 



ting denote the cokernel of the injective map 0(— A) Ox such that (^0 = ad(a) coincides 
with the composite F*E ©(—A) Ox, there is an exact sequence 

0(A) det Eo Eq ^ Ox kA ^ ^ 

of coherent sheaves over X and the Probenius pull-back of the exact sequence (13.2(1 induces an 
exact sequence 

^ C(A) ® det^o ^Bi^ F*Qe ^ A;a ^ 

Since the composite 0{A) (gidet-Eo Bi cox defined at (13. 6|) is non zero, it is injective and 
B2 identifies with a subbundle of coker(/9i), hence of F*Qe. Now, the quotient coker(/9i)/i?2 is 
also the cokernel of the maps defined in (|3.6|) and the lemma follows. □ 



It follows from the filtration (12. 3|) of Bq and from the previous lemma that there is an exact 
sequence 

^ ^ F*Qe/uj''x^ ^ k^^^^E) ^ 

Lemma 3.15. When p > 2g and for any {E, a) in TL, the quotient sheaf F* Q e / uj^ ^ is torsion 
free. 
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Proof. Assume that the torsion D := Tots{F* Q e / uj^ ^) of the quotient sheaf F*Qe/u)^ ^ is not 
zero. Then the kernel of the composite map 

can be written under the form co^ ^ (Si 0{D) where 0{D) is a hne bundle of degree deg-D > 
over X. By functoriality of relative duality, the map F^Ox Qe factors through the surjective 
map F^O{D) Qe- But the remark \T2\ following Lemma [2?T] tells us that 

r^.. (p+l)(ff-l) + degp) (p-l)(g-l) 
Up-2{Om = > — ^ 

and this is a contradiction. □ 

Therefore, if dimHom(u;^^^, F*Qe) > 2, there is an injective map w^"^ ©w^^^ ^ F*Qe and 
the kernel of the composite 

cj^"^ © w^"^ ■-^ F*Qe k^^{E) 

identifies with a rank 2 subbundle of B2 with slope at least {2p — 3)(g — 1). But B2 has a 
unique such subbundle, namely -Bp_2 (see Subsection 12.31 again) which is the unique non split 
extension of by Therefore, it cannot be a subbundle of w^"^ © '^x"^ ^^^^ ^ 

contradiction. □ 

3.4. Fitting ideals. Recall from the GIT construction of the moduli space that there is a 
Quot-scheme Quot, quasi-projective over k, that represents the functor from the category Sch/k 
of schemes over k to Sets defined by 



k'^ (g) OxixT ^ Qr © q*Xj_L>o 



Qt is locally free of rank p — 2 and, 

Vt G T{k), Qrit) is stable } / 

and has degree {p — l){g — 1). 



where L>o is a suitably high degree line bundle over Xi. We require that there is an isomorphism 
k^ © Ot — > <1t,*{Qt © Qxi-^>o) ^^^^ identifies a surjective map k^ © OxixS Qt © Qxi^>o 
to the evaluation of global sections of Qt © g^^-L>o. This gives a natural action of PGL(A^) 
on Quot (resp. on the universal quotient bundle Qquoi over Xi x S) and those actions are 
compatible. The coarse moduli property associates to the universal bundle Qquoi a morphism 
vr : Quot Af^i which makes Quot into a locally isotrivial principal PGL(A^)-bundle over the 
stable locus {M^^^ of M^^. 

Remark 3.16. If U is an affine open subset in (M^J^ tt-'^{U) is an affine open subset of Quot 
and there is an isomorphism 

In particular, any PGL(A^)-invariant sheaf of ideals over Quot descends to a sheaf of ideals over 

(M«)^ 

For any T-point Qt '■ T — > Quot., we let 

2^ :=Fitt((p_2)2„2)(s-i)(^^9T,*(Wom(F,Ox, Qt))) C Ot 
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denote the {{p — 2)^ — 2){g — l)-th Fitting ideal of the first direct image of Hom{F^Ox , Qr), 
where Qt also denotes the vector bundle QxiQQuot)- Because Xi is a curve, the formation of 
R^qr,* commutes with any base change. It is also the case for the formation of Fitting ideals 
(see [E[ Corollary 20.5]), and there is a canonical isomorphism 3t — QTC^Quot) compatible with 
base change. As a consequence of the construction, we obtain the following 

Lemma 3.17. The Fitting idealJQuot is PGL{N) -invariant, hence descends to an sheaf of ideals 
3 over . 

One checks that the sheaf 

:= qT,*nom{q*x^{F,Ox^uj^l), Qt) 

is locally free of rank {g — — 2) and, assuming that iiom.{F^Ox, Qrit)) = for a generic 
point t in T, one obtains a projective resolution 

(3.8) O^T^^T^^ R^qT,*Hom{q*x^F,Ox, Qt) ^ 

oi R^qT,*Hom{p*j^^F^Ox, Qt), where 

jtO 0®2(g-l)p(p-2) 

is a locally free of rank 2{g — l)p{p — 2), by taking a non zero global section of coxi- Notice that if 
p > 2g, it is the case that a generic Q in (M^J* does not lie in the closed subscheme H. Indeed, 
it follows from Corollary 13.21 and Lemma 13.81 that 

dimi^ < dimSx^ + g - 1 = 5{g - 1) - 1 < {p - 2f{g - 1) - 1 

In particular, there is a projective resolution as above when T = Quot. 

Proposition 3.18. Assume that p > 2g. Then, 

(1) the Fitting ideal 3^ is the defining ideal of the closed subscheme Ti in M^_^ ; 

(2) if Ti. has the expected dimension 2{g — 1), it is locally complete intersection (hence Cohen- 
Macaulay) and equidimensionnal; 

(3) if (E, a) is a point of Ti such that dimHom(S, Qe) = 2{g — 1), then Ti. has the expected 
dimension at E and it is regular at this point. 

Proof. Since for any Q in M^^, a non-zero map F^Ox — > Q is surjective (use Lemma [27T]l . it 
is clear that Ti and the zero locus Ti' of J*^ have the same support. To prove that there is a 
scheme-theoretic isomorphism = 7"^', it is enough to prove that 7i{T) = Ti.'{T) for any affine 
A;-scheme T (the inclusion ?i{T) C Ti.'{T) is clear from the definitions). 

Assume that Qt is a T-point of that cancels the Fitting ideal J'^ . Using the correspond- 
ing morphism to the Quot-scheme Quot (still denoted by Qt), one pulls-back the projective 
resolution of R^ qQuot,*H om{q\^F^,0 x , Qquoi) and obtains an exact sequence 

Q-T{^Quot) ^ QTi^Quot) - R\T,.Hom[q*x^F,Ox, Qt) - 
Using Proposition 13.131 and its proof, we find that R^pT,*Hom{p'^^F^Ox, Qt) is locally free of 
rank {{p - 2)^ -2){g-l) + l and the kernel of QTi^Quot) ^ QU^Quot) is locally free of rank 1. 
Locally on any sufficiently small open subset U of T, one can therefore find a non-zero morphism 
f3u : Px.^F^:Ox Qu which is surjective since it is so at any closed point of U (use Nakayama's 
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lemma). Since is a closed subscheme of Mj^^ and since T is affine, the Pu can be glued and 
we conclude that Ti'iT) C TLiT). This proves (1). 

Assume that TL has the expected dimension 2[g — 1). At a point {E, a) of choose an afhne 
neighborhood ?7 of t in (M^^)* and a finite etale cover U ^ U that trivializes the PGL(A^)- 
bundle vr : Quot — > (M^J*. Choose a section U Quot and let Qjj denote the corresponding 
pull-back of the universal quotient bundle Qquoi over JJ . Also denote by Tifj the inverse image 
of H. in U. It is locally complete intersection and since [M^^Y (hence U) is smooth, Tif) is 
Cohen-Macaulay (see, e.g., [Hal Proposition II. 8. 23]). Because it is defined as a determinantal 
variety, one knows (see, e.g., |Ful Chapter 14]) that each irreducible component ofH(j (hence of 
H) has dimension at least 2{g — 1), hence equal to 2{g — 1). All these notions go through etale 
descent (see the discussion in fE', p. 466] for locally complete intersection) and this proves (2). 

Assume that dimHom(S, Qe) = 2{g — 1). Since Hom(£', Qe) is isomorphic to the tangent 
space of TC at {E, a), the dimension of TC at this point is at most 2{g — 1). Since W is a 
determinantal variety, its dimension is at least 2{g — 1). Now, (3) follows from standard facts on 
Quot-schemes of the expected dimension. □ 

Remark 3.19. Notice that one can do the same kind of reasoning for the image of in A^^i 
that is characterized by Hom(£', F^Ox) 7^ 0. This proves in particular that if E' is a stable 
rank 2 subbundle of F^^Ox such that F*E = Ox ® Ox, its image in Nx^ is a singular point 
of the image h£{H) (see [Fuj again). By constrast, if {E, a) is a stable point of 7i such that 
F*E Ox © Ox (see Subsection 14.21 for the existence of such a point), applying the functor 
HoniXjiE, — ) to the exact sequence (|3.2p yields an injection 

T^E,a)'H = ^omx,{E, Qe) H\Xi, £nd{E)) ^ TeNx, 

and the morphism hs is locally (in the neighborhood of a point in 7"^ \ 0.2) a closed immersion. 

4. Distinguished loci 

Theorem 12.51 applies and it follows from a result of Nakajima (see, e.g., [Zhj) that X is even 
super- ordinary is the sense that all of its abelian Galois etale covers are ordinary. 

4.1. The semi-stable boundaries of 7i and Ti. If a strictly semi-stable rank 2 and degree 
vector bundle, it has a degree line subbundle M. Now, if is a subbundle of F^Ox, so is 
M and the latter has order dividing p. Upon twisting by a suitable element of G, we can 
assume that E admits Oxi as a line subbundle and because of the exact sequence (11.11) . the 
quotient bundle EjOx^ identifies with a degree line subbundle of the bundle B of locally exact 
differential forms. 

If L is a point of 0^ and L ^ is the corresponding injection, we define E^ as the inverse 
image of L in F^Ox via the canonical surjection F^Ox B. There is an extension 

(4.1) ^ Oxi ^ E^ ^ L ^ 

(that also depends on the inclusion L ^ B) and Qel is canonically isomorphic to the quotient 
sheaf 

Ql := Ql^b = B/L 
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fitting in the exact sequence 



(4.2) 



Using the Quot-scheme Hb defined in Subsection 12.31 this defines closed immersions Hb ^ 
and Ti.B H/G and Hb identifies in particular with the semi-stable boundary of H/G. Thus, 
the semi-stable boundary 7^^** of Ti identifies with a degree 2^9 etale covering of TIb- 

Remark 4.1. Notice that the adjoint map F*E^ Ox provides with a splitting of the Frobe- 
nius pull-back of the exact sequence gU). In other words, F*E^ ^ Ox ® F*L. 

4.2. Vector bundles arising from Prym varieties. We recall some basic facts about Prym 
varieties associated to double etale covers of X and their relation with rank 2 vector bundles 
invariant under the action of an order 2 line bundle over Xi. The reader can look in |Mu] for 
proofs and details (see also [Duj where we use these technics in the same context). 

Choose an order 2 line bundle r over X and consider the corresponding double etale cover 
a : X := Spec{Ox (B t) ^ X. To this cover is classically associated the homomorphisms 
a* : Jx ^ Jx (with finite kernel isomorphic to < r >) and Nm : ^ Jx defined by 
M ^ M i*M (where i : X ^ X is the involution permuting the sheets of the double cover 
a). The latter is surjective and its kernel is the product of a finite group isomorphic to (Z/2Z) 
and of a principally polarized abelian variety Pr (the Prym variety associated to a) of dimension 
g — 1. The composite Nm o a* is [2] : Jx ^ Jx- 

For any L in Jj^, a*L is a rank 2 vector bundle with determinant Nm(L) ® r, satisfying 
a*L(8)T = a*L. It is stable if L does not belong to im(a*) and a^a*L = {Ox ®t)®L for any L 
in Jx- Conversely, if E is any vector bundle of rank 2 and degree 0, an isomorphism E ®t = E 
allows one to give E a structure of {Ox © r)-free module of rank 1, meaning that E = a^L for 
some L £ Jj^. In particular, there is a morphism Nm~^(r) — > Sx defined by L i-^ a*L that 
surjects onto the r-invariant locus of Sx which identifies with the intersection of Sx with the 
(disjoint union of) two projective spaces in |20| that are invariant under the action of r. It is 
easily checked that, restricted to an irreducible component of Nm~^(r) (that identifies with Pr), 
the morphism Pr — > Sx factors through the Kummer variety Kp^ := Pr/{±1} of Pr and that 
the latter is embedded in Sx by this map. In particular, it intersects the strictly semi-stable 
locus Sf = Kx in 22(9-i) points that identify with the 2-torsion ^^[2] = Prf]im{a*) of Pr- 

Since p is odd, F induces an isomorphism Jxi[2] — > Jx[2] and letting n denote the image of 
r, the Frobenius inverse image ai : Xi ^ Xi of a : X ^ X is the etale double cover associated 
to Ti. We choose a morphism Sxi associated to r. Following [SGAH Lemma 1. 11], one 

has a cartesian diagram 



X 



F 



a 



X 



F 
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and the action of Probenius on vector bundles arising from the Prym variety P^. is given by 
multiplication by p on Pr. . 

Proposition 4.2. Let ti be any order 2 line bundle on Xi and let {E, a) be a stable Ti-invariant 
point of H. Then, one has dimHomxi(-E, Qe) = 2(5 — 1). 

The scheme-theoretic locus of Ti-invariant points of H is finite reduced and contains exactly 
22(9-1)^1 (p9 - 1) points. 

Proof. A point {E, a) in Ti such that E ®ti = E can be written under the form ai^ *L for some 
line bundle L in Jj^^ , and the exact sequence (13. 2p becomes 

Use adjunction and relative duality for the map ai and the commutativity of the diagram 
preceding the statement to obtain an injective map L —> F^Oj^ and deduce the fact that L 
has order p (since E is stable, L ^ O^^. Since F*ai^^,L = a^Ox — Ox © r, a ri-invariant 
point of H neither lies in the base locus of V nor in H2. Since Nm(L) also has order p and since 
[2] : Jxi induces an isomorphism of Jxi[p], we can assume, upon twisting by a suitable 

order p line bundle over Jx. , that Nm(L) = Ox. ■ 

Pull the exact sequence above back to Xi to obtain the exact sequence 

and twist the latter by L. Consider the following commutative diagram of Oj^^ -modules with 
exact rows and columns 















i 


i 








Ox. 










i 


i 










^ F,0^ - 




- 





I 


i 










B - 




- 





i 


i 





















0^ o 
^ 



where the middle column is the exact sequence for Xi. Use projection formula for ai 

and the isomorphism ai, *L = (ai^*L)^ (deriving from the assumption Nm(L) = Ox) to obtain 
an isomorphism ai^*(aiQai,L L) = Homo^^{ai^^:L, Qai^i)- The long exact sequence of 
cohomology associated to the bottom line thus yields a surjection 

H\Xi, B) ^ Ext3,^(ai,*L, Q„, ^l) 

Because X is a general curve, it is super-ordinary and its double etale cover X is ordinary. 
Therefore, Ext^^(ai^*L, ^l) = and the dimension of Homxi (ai, Qa. ,l) is given by 
Riemann-Roch. Namely, one has 

dim(Homxi (ai, *L, Qa.^ .l)) = 2(p - 2) [- — ^"^^^ - g + 1 
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Now, assume that {Et, «t) is a T-point of TL for some connected /c-scheme T . Assume that 
there is an isomorphism £t x over Xi x T. Then, there is a morphism T —* P^^ 

Li — >ai *L 

such that the composite T Pr^ '■ — > Nx^ coincide with the composite T ^ 7i — > Nxi ■ 

This morphism T P^-^ factors through the reduced subscheme i-VilpJred and it proves that the 
Ti-invariant locus of Ti is finite and reduced. Since Ox^ © ^i cannot be imbedded in F*Ox, we 
find that the intersection of H and of the image of P,-! agrees with the stable part of the image of 
Pt-i[2p] endowed with the reduced induced structure. Hence, it is in one-to-one correspondence 
with the set 

(P.j2p]\P,j2])/{±} 

whose cardinality is (22(f-i)p3 _ 22(9-1) )/2 = 22(9-i)-i(p£' _ l). □ 

Applying Proposition 13.181 (3). one has in particular 
Corollary 4.3. If p > 2g, Ti is smooth at any Ti-invariant point. 

4.3. The set of Frobenius trivialized points of H. The set of (integrable) connections on 
Ox © Ox is affine under B.omx{Ox © Ox, (Ox © Ox) (X) u)x) and we use the trivial connection 
induced by the isomorphism F*(C'xi©Oxi) = Ox®Ox- The group Autx(Ox©C'x) = GL(2, k) 
acts on the vector space Homx(Ox © Ox, (Ox © Ox) 'S'UJx) by conjugation (we will talk about 
transport) and the vanishing of the p-curvature is invariant under transport. 

Since we will perform some explicit computations, let us recall the following general technic. 
Being given a rank r bundle M with connection V over a curve X, choose an affine open subset 
U of X that trivializes M and ujx, choose a differential form ujq (defined over U) that does not 
vanish over U and provides with a trivialization of cox- Choose a germ of derivation defined 
over U such that ^o('^o) = 1- The restriction of V to [/ is given by a r x r matrix T with 
coefficients in T{U, Ou) such that 

V(/ ®e)= T{e) 0ujo + e0df 

The vanishing of 

^v{eo) = {T + 9o)^-9l{u;o)T-eP, 

suffices to prove the vanishing of the p-curvature. Computing (T -|- 9q)p recursively and looking 
for (r + ^o)" under the form ELo^i^^^O' one finds 

n 

(4.3) (T + 0o)"+' = {T + 0o)To^"^ + Yl ((^0 + T)r^") + r,^'!^) 9^^ + T}r^9^+' 

k=l 

By definition, one has Tq^^ = T and T^^^ = 1. This implies that T^""* = 1 for all n > 1 and we 
set Tq^^ = 1 for consistency. One checks by induction that 

Tit = ( r ) ^0'^ all n, < r < n, 
and consequently, that il^\/{9o) = Tjf^ — 9q{ujq)T. 
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Remark 4.4. (1) The explicit computation of Tq is usually very tedious since T and do not 
commute. Still, in the case of a line bundle, the expression dramatically simplifies to give 

^^{9o) = TP + 9P,-\T)-e^,{ujo)T 

(2) In case M has trivial determinant, the explicit computation of the determinant connection 
ensures that V'^''* is represented by Tr(T). 

Let us look at a point E of H such that F*E = Ox ® Ox- If E is stable, it is a point of 7^2- 
By contrast, assume that E is & strictly semi-stable vector bundle that one can think of as an 
extension 

^ Oxi ^ E^ ^ L ^ 

with L in Qb (see Subsection |43|) . Since detF*^^ ^ F*L ^ Ox, L has order p and as X 
is a general curve. Theorem 12.51 (3) applies and such a point L is a regular point of @b- In 
particular, dimHomxi(^, B) = 1 and E is the split vector bundle Oxi ®L (notice, though, that 
V'o ■ © Ox Ox maps a local section (/i, /2) to the sum fi + /2 rather than to one of the 
/j's, and that the second fundamental form is indeed non zero). Upon conjugating by a global 
automorphism of Ox © Ox , there is thus a canonical p-integrable connection 

^can _ f d \ 
\0 d + UL J 

on Ox © Ox, where is the unique global differential form such that d + lol '■ Ox has 
vanishing p-curvature and such that the corresponding line bundle on Xi is L. Let Ul be the 
affine subset of X where ojl does not vanish and let O^he the germ of derivation (defined on 
Ul) such that Ol{ujl) = 1. Because the connection d + uiL has vanishing ]5-curvature, one finds, 
using the p-curvature formula for line bundles in Remark 14.41 that 

i^d+^.m = {Ol + ly - oliujL) -01 = I- elicoL) = 

If X is any regular function defined on Ul, we let ^^(a;) denote the regular function 9L{dx), 
where d : Ox ^x is the canonical derivation. 

Lemma 4.5. With the notations above, let uj be any global differential form over X such thattOL 
and UJ are linearly independent and denote by x the non constant rational function u/ujl- Then, 
none of the expressions 

Y^elix) and'Y,(^l^)el{x) 

k=l k=l ^ ^ 

vanishes. 

Proof. By assumption, dimHom(L, B) = 1 for all line bundle L of order p on Xi and the map 

Ext^(L, OxJ — ^ — ^ Ext^(C'x, Ox) is injective. Therefore, if a S is S-equivalent to Ox^ © L 
and satisfies F*E = Ox © Ox, it is necessarily isomorphic to Oxi © L. In terms of connections, 
it means that both connections 
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on Ox ® Ox have non-zero p-curvature since none of them is transport equivalent to Vcan. 
Because 

X \ f f \ f X \ f f I X \ f 1 f \ (If 



for any rational function / on X, we compute by induction (using the expression (|4.3p ) that 

^y{OL) = ( I n=i^^i(^) ^ 1^ resp. Vv'C^l) = 1^ ° ^^=' k ^ '''' 

and conclude the proof. □ 
Proposition 4.6. The set Ti.2 (resp. H2) is proper. 

Proof. Assume that the contrary holds. Then there is a point of type Oxi ©-^^ as above that can 
be continuously deformed into a point of 7^2- In particular, letting denote X x^. Specfc[e]/e^, 
the connection V'^'^'^ above has a non-trivial linear infinitesimal deformation Vg := V'^'^'^ -|- eT 
over Xs (with T in Romx{Ox © Ox, {Ox © Ox) ® ujx) = F°(a;x)®'^)) whose p-curvature 
vanishes and whose second fundamental form is non-zero (and remains so after conjugation by 
any element of Autx,(Ox, © Ox,) = GL(2, k[e]/e'^)). 

Because X is ordinary, the kernel of Vj : Jxi Jx is reduced and such a deformation 
necessarily induces a trivial deformation of the determinant connection. It means that T is a 
traceless morphism (see Remark 14.41 (2) above) that we write under the form 

-21 -11 J = 1 -21 J-^i^^O 1 J=^--n^ 

Write for the function Wi/wL, i = 11, 12 or 21. It is regular over Ul. Let denote 
Ve + euoiil = V""" + eT'. It follows from [Ossl Corollary 3.6(iii)] that one has 

(4.4) V'v.(^l) = V'v^(^l) + e((/(''))^ + er'(/''^) - (/^"^))^ 

where we use 6^{u)l) = 1 for the last term. 

Introduce the matrices "^^^q -^0"''' where Rq^^ = R = ^ "^f^"^^^ ^0 

and where i^Q™^^^ = {9l + J)Rq^^ + RJ for all n > 1. One checks that 

{eL + J + eR)iJ + eR^^^) = J + ei?[,"+^^ 

(n) ( 2^") fl^^) \ 

and writing Rq under the form I ^21) " 1 > we check by induction on n > 1 that 

\ fn ^ / 

^(11) ^ ei-\f^^^)) 

n-l 

fT^ = E^lif^'"^) 

k=0 
n-l 



k=Q ^ ^ 
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Using (|4.4p and the fact that iP\"^{9l) = ^{Ro — R), the vanishing of the p-curvature iI^\7{6l) 
yields that Z*-^^^ and that both 

k=l k=l ^ ^ 

are 0. Using the lemma above, both 1012 and UJ21 are colinear with ujl and Vg is conjugate to 
the trivial deformation by some constant automorphism of Ox^ © Ox^ ■ This is a contradiction 
and it proves the proposition. □ 



5. The genus 2 case : Proof of the theorem 11.11 

In this section, we will assume that p > 5 (recall that [LP21 Section 6] gives a complete 
description of H in characteristic 3). 

In genus 2, H has dimension 2 since it is also defined as the residual divisor of Kxi in V^^{Kx) 
(notice that since X is ordinary, the multiplicity of Kxi in V^^{Kx) is 1). It immediately 
follows from the proposition 14.61 that the set H2 is finite (it is proper in the complement of an 
hypersurface in P^). Also, the base locus of Verschiebung is finite and, as a consequence, the set 
of 1-dimensional fibres of the morphism h : H ^ H is finite (see Lemma [3?8]) . Therefore, Ti also 
has dimension 2 and it is locally complete intersection and equidimensional (Proposition 13. 181) . 

5.1. The semi-stable boundary of 7i in genus 2. A generic point {E, a) in 7i gives rise to 
an exact sequence 

^ F* det ^ ^ F*E ^ Ox 
Push it forward by Frobenius to obtain an exact sequence 

^ detF F^Ox -^E® F^Ox ^ F^Ox ^ 

and notice that the composite E > E (g) F^Ox F^^Ox, agrees with a. In other words, 

there is a commutative diagram with exact rows and columns 



1 i 

det E^F^Ox = det E^F^Ox 

I I 

(5.1) 0^ E ^ E(g)F^Ox E®B ^0 

II i i 

0^ ^ ^ F^Ox Qe ^0 

i i 


It follows from Remark 13.31 that if is a stable bundle in H with (det£^)^ 7^ Oxi, then F*E is 
split (see also [LP 11 Remark 6.2] for another argument involving the hyperelliptic involution). In 
particular, Hom(£^, det E<^ F^Ox) is non zero for such a point (hence for all by semi-continuity). 
Applying the functor Hom(i?, — ) to the rightmost column in the diagram above, one finds that 
H^{£nd{E) ® B) is non zero for all point of Tl. 
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Lemma 5.1. The set defined as 

^ _ 1 rank 2 semi-stable vector bundles E with trivial 1 
I determinant over Xi such that H^{£nd{E) / J 

is compatible with S-equivalence. It defines a closed subset of Sxi whose intersection with Kxi 
agrees with the support of the divisor on Kxi induced by the totally symmetric divisor [2]~^{@b) 
on Jxi ■ 

As sets, there is an inclusion H CT,b- 

Proof. Take a strictly semi-stable vector bundle E with trivial determinant written as an exten- 
sion E ^ M ^ where M is a degree line bundle over Xi. We tensor this exact 
sequence by E ® B, M (S) B and (g) B successively and take the associated exact sequences 
of cohomology spaces. Because X is ordinary, H^{B) = H^{B) = and one finds an exact 
sequence 

^ H^{M-'^ ®B)^ H°{£nd{E) (8) B) ^ H^{M'^ ® B) ^ H^{]Vr^ (g) B) 

On the other hand, £nd{M M'^) = Of ^ M'"^ M^. Because the divisor Gs is symmetric, 
we find that H^{£nd{E) i?) 7^ is and only if is a point on 0s, or equivalently, if 
H^{£nd{M M"-*^) i?) 7^ 0. Because Qb is totally symmetric in the sense of Mumford, it 
descends to a divisor on Kx^^ with support the set 

{[M © M-i] G Kx^ g Bb} 

The last assertion just restates the conclusion of the discussion preceding the statement. □ 

It follows from the Theorem 12.51 (1) that is smooth and that the same holds for TIb — &b- 
The inverse image of 0_b via the multiplication map [2] : Jxi Jxi is also smooth (and 
connected). Since the ramification locus of the Kummer map Jxi — »■ Kxi is Jxi [2] and since 
does not go through the origin, we find that the divisor in Kx^ induced by [2]^^0b is connected 
and smooth, hence irreducible, with support TibCI^Xi- 

Corollary 5.2. The semi-stable boundary Hf]Kxi agrees with T,Bf]Kxi- In particular, it is 
irreducible 

Proof. We already have the inclusion of sets Hf]Kxi C Y,b Cl^Xi- Conversely, take a line 
bundle M over Xi and assume that lies in 0^. Then, the inverse image of via the 
canonical surjection F^,Ox — > B provides with an extension — > Oxi — E'^'^^ — > — > and 
twisting by M~^, we obtain an exact sequence 

^ M'^ E^^^ M-^ ^ M ^ 

that is non split as soon as M^P / Oxi- By definition, this is a point of H and [M M~^] lies 
in H. □ 

Proof of the Theorem Any irreducible component of H is mapped onto a (irreducible) 2- 
dimensional closed subscheme in H. In particular, it necessarily meets the semi-stable boundary 
Kxi along the 1-dimensional irreducible closed subscheme Hf]Kxi. The same holds in Tl. 
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Namely, any irreducible component of Ti meets a irreducible component of the (smooth) semi- 
stable boundary 7^** and the latter identifies with a degree 16 etale cover of Hb — &b- The 
composite He ^ T~i Nx-^ — Jx^ identifies with the composite He — > 0_b C Jxx and it 
follows from the cartesian diagram (|2.ip and the connectedness of the inverse image [2]~^6b 
that 7^^* is connected, hence irreducible. As a consequence, Ti is irreducible and so is H . 

Also, if the singular locus of H had dimension 1, it would necessarily intersect H{^Kx^ and 
this is a contradiction. Since H is complete intersection in it is normal. The fact that it 
has degree 2(p — 1) is a straightforward consequence of the proposition 14.21 and the equality of 
divisors follows since V^^{Kx) has degree 4p. □ 

5.2. The hyperelliptic involution. In the previous subsection, we used the Remark 13.31 to 
derive the fact that, for a generic point {E, ^, a) of 7^, there is also a point {E, a') in Ti. 
More generally, recall that the hyperelliptic involution i of Xi changes a degree line bundle over 
Xi to its inverse. For any T-point £t F^^^t oiTi, we define the T-point {i*{£t)^ , licxT)) 
of 7i as the composite 

This defines an involution of H (still denoted by z) whose fixed locus consists in the closure of 
the set of points {E, r, a) such that = Ox and such that a induces an exact sequence 

F*E ^ r ^ 

We let [H] denote the quotient of Ti. under this involution and we notice that h : H ^ H 
(resp. <I>i : 7^ ^ Bl2(Jx)) induces a morphism [h] : [H] H (resp. [$i] : [7^] Bl2(i^x), where 
^hiKx) denotes the smooth K3-surface obtained as the blowing-up of Kx along its singular 
locus). It immediately follows from the discussion above and from Lemma [3^81 that the map 
[h] : [7^] —I- H identifies set-theoretically with the blowing-up of H along the union of the finite 
sets consisting in the base locus of V on the one hand and in the points of H2 on the other hand. 
The exceptional line above a point in the former set canonically identifies with Xi/i = \u!Xi \ — IP^ 
whereas the exceptional line above a point in the latter set is isomorphic to PHom(£', F^Ox) 
since it lies in the fixed locus of i. Since H is normal, Zariski's main theorem readily gives the 
following 

Proposition 5.3. The morphism [h] : [7^] H identifies with the blowing-up of H along the 
union the finite sets consisting in the base locus of V on the one hand and in the points of H2 
on the other hand. 
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